Connectionist neuroscientists and computer scientists have long believed that the structure of biological neural networks is central to brain function. Conceptually, it is simple to find the structure of the neural networks in the brain. One needs to trace the “wires” of neurons to understand their connectivity. Unfortunately, the problem is complicated by the fact that axons can be as small as 100 nm in diameter and parts of dendrites known as spine-necks can be as narrow as 50 nm (K. L. Briggman, et al., “Towards neural circuit reconstruction with volume electron microscopy techniques,” CurtOpin Neurobiol, 16(5):562-70, 2006. 3). This requires the complete reconstruction of densely packed and intertwined, highly branched processes whose sizes approach the resolution of this electron microscopy (EM) technique, greatly exceeding the capability of conventional segmentation methods. Due to the large degree of branching, neurons have extremely large amounts of boundary compared to their volume. Even very small error rates in boundary detection can thus ruin the ability to segment such objects. Further, the enormous volume of data to be segmented imply that small differences in performance can imply differences in weeks to months of human effort to proofread machine segmentations.
While there has been much research on the image processing and segmentation of natural images, little work has been done for EM. Consequently little is known about good features for affinity graph generation for these images. In contrast to natural images, EM images contain virtually no region-based segmentation cues. The interior of one neuron essentially looks the same as the interior of a neighboring neuron, and yet must be segmented as a different object. Thus, edge-based features are likely the most useful. Given the lack of region-based cues, it is fortuitous that three dimensional (3D) EM images lack the problems of occlusion that are common in natural images.
Prior work on electron and optical microscopy reconstruction has either been on isolated stained neurons (in essence a binary segmentation problem) (J. C. Fiala, “Reconstruct: a free editor for serial section microscopy,” Journal of Microscopy, 218(1):52-61, 2005. 3) or has required significant human intervention to complete (Y. Mishchenko, et al., “Automated 3d reconstruction of neuronal circuitry from serial electron micrographs,” In Society for Neuroscience, 2005. 3, J. White, et al., “The Structure of the Nervous System of the Nematode Caenorhaklitis elegans,” Phil. Trans. of the Royal Society of London. Series B, 314(1165):1-340, 1986. 3). Most attempts have involved hand-designed filtering architectures with little machine learning (A. Vasilevskiy, et al., “Flux maximizing geometric flows,” Pattern Analysis and Machine Intelligence, IEEE Transactions on, 24(12):1565-1578, 2002, K. Al-Kofahi, et al., “Rapid automated three-dimensional tracing of neurons from confocal image stacks,” IEEE Trans. Info. Tech. Biorned., 6(2):171-187, 2002, E. Jurrus, et al., “Axon tracking in serial block-face scanning electron microscopy,” In Workshop on Microscopic Image Analysis with. Applications in Biology, 2006).
Convolutional networks have achieved a great deal of success in high-level vision problems, such as object recognition (Y. LeCun, et al., “Gradient-Based Learning Applied to Document Recognition,” Proceedings of the IEEE, 86(11), 1998, Y. LeCun, et al., “Learning methods for generic object recognition with invariance to pose and lighting,” In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2004), 2004. 2, 3). In the past, convolutional networks have been successfully used for image processing applications, such as object recognition and handwritten digit classification (Y. LeCun, et al., “Backpropagation applied to handwritten zip code recognition,” Neural Computation, 1:541-551, 1989. 2, 4, 8. For these problems, the convolutional networks are trained to produce a categorical classification, such as the identity of the digit.